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A lege INTRODUCTION 

In the current period of fiscal restraint, the acquisition 
of weapon systems has come under ever increasing scrutiny. 
Costs of these systems are influenced by performance 
characteristics of the system and schedule demands placed on 
the acquisition process. This thesis will review how the 
acquisition community can trade off performance for cost and 
trade off schedule for cost so as to stay within the binding 
budget constraints that will exist for procurement of major 
weapon systems. 

The primary focus of this thesis is to conduct a 
literature review of studies and models which have been 
performed in the past and to relate those to current and 
future acquisitions. Specifically, this thesis attempts to 
investigate how the defense acquisition community can better 
use these studies to make more informed decisions regarding 


the potential costs of particular tradeoff options. 


A. BACKGROUND 

The changing military threat to the United States and 
increased attention to domestic issues have brought the 
Department of Defense budget under considerable pressure. 
Reductions in the budget of twenty-five percent by fiscal year 


1995 are a certainty, and it is all but certain that further 


reductions will be imposed. This declining budget environment 
forces the Department of Defense, and particularly the 
acquisition community, to make decisions on weapon systems 
which will have lasting impact on both future budgets and 
future capabilities. The President has highlighted this fact 
with the announcement in his 1992 State of the Union address 


Ehat: 


e The B-2 program will be terminated at 20 aircraft. 
°e The Small ICBM program will be canceled entirely. 


e Production of the W-88 warhead for Trident II SLBMs will 
be terminated. 


e Purchases of the Advanced Cruise Missile beyond those 
already authorized will cease. (Cheney, 1992, p. 20) 
These cuts are in addition to the over one hundred weapon 

system program terminations which have occurred in the last 
two years. The cuts taken together amount to a $50 billion 
Savings in fiscal years 1993 through 1997. (Cheney, 1992, 
Ds ss) Unfortunately, decreasing or canceling procurements 
will, over the long term, lead to what Chairman of the Joint 
Chiefs, General Powell, called a "hollow force." It is for 
this reason that the Secretary of Defense stated before the 
House Budget Committee that: 

The military technological revolution will continue to 
pose challenges to our forces both to keep up with 
competing technologies and to get the greatest potential 
from the systems we have; and, that a continued and 


substantial research and development effort...will be 
required to maintain our advantage. (Cheney, 1992, p. 17) 


Given that continued monies must be spent on new weapon 
systems, the question then becomes how best to spend those 
monies and what can be gained or lost by trading off schedule 
considerations and performance parameters against limited 


budgets. 


B. OBJECTIVE 

The relationship between cost, schedule, and performance 
parameters for a given weapon system 1s a complicated one. 
Generally speaking, cost and schedule and cost and performance 
are directly linked while the schedule versus performance 
parameters are indirectly linked through cost. One can easily 
conceptualize the link between cost as the dependent variable 
and schedule as the independent variable if the performance 
parameters are held constant. In this situation, as schedule 
increases, overhead costs over the longer schedule will tend 
to drive costs up. Alternatively, if the schedule is 
compressed, more work will have to be accomplished in less 
time and again it is likely that the costs will increase. 
When the schedule is held constant, achieving "better" 
performance will undoubtedly cost more. These relationships 
imply that there is at least one point where cost is minimized 
for a certain schedule and set of performance parameters. In 
theory, this point is established as a result of the contract 
between the government and the contractor. Moving away from 


this point in any direction usually entails trading off the 


design, cost, or delivery of the particular weapon system. 
The objective of this thesis is to investigate the conceptual 
foundations of models which attempt to quantify the tradeoffs 


between cost, schedule, and performance. 


C: SCOPE AND LIMITATIONS 

As previously mentioned, the scope of this thesis is to 
investigate, through a detailed review of the literature, what 
studies have been performed with respect to the tradeoffs 
between cost, schedule, and performance characteristics in the 
acquisition of weapon systems. The thesis attempts to answer 
the questions: What studies have been performed?; What are the 
cost estimating relationships (CERS) involved?; and How can 
the defense acquisition community use these studies to make 
informed decisions regarding particular tradeoff options? 

This thesis is not intended to question strategic planning 
policy nor predict future funding levels or force structures. 
Nor does it attempt to measure cost versus effectiveness of a 


particular weapon system. 


D. RESEARCH METHODOLOGY 

Computerized literature searches were the primary method 
for collecting information and models on the effects of cost, 
schedule, and performance on weapon system acquisition. 
Searches were made of the Defense Logistics Studies 


Information Exchange (DLSIE), Defense Technical Information 


Center (DTIC), and the National Technical Information Service 
(NTIS). A secondary method was a personal review of the Rand 
Corporation yearly Indexes from 1946 through 1990. Keywords 
used included combinations of the following; cost estimating 
relationship, CER, weapon acquisition, weapon systems, cost 
and performance, cost and schedule, and tradeoffs. The 
references of relevant works found by the above methods were 


also examined to complete the search. 


E. ORGANIZATION OF THE THESIS 

In order to better understand the trends that weapon 
system procurement has followed, Chapter II provides a brief 
discussion of the acquisition experience of the last few 
decades. It also highlights some of the more important 
foundations of cost estimation and their applications to 
weapon systems. The remaining chapters are dedicated to 
discussion of the various cost, schedule, and performance 
tradeoff models, problems with their use and potential for the 
future. 

Many equations with complex variables are presented in 
this thesis. The confusion which Surrounds the variables has 
been minimized by presenting definitions in the body of the 
thesis adjacent to each equation. It is hoped that the reader 
will find this method more acceptable than if the definitions 


were listed together in an Appendix. 


Appendix A contains selected regression equations by 
Alexander and Nelson for aircraft turbine engines which 
supplement those in the body of the thesis. Appendix B 
contains Bureau of Labor Statistics based price indices for 
the years 1946 through 1973. Appendix C provides an annotated 
bibliography which provides references in addition to those 
cited in the body of the thesis which have either 
theoretically or quantitatively described the relationship 
between or among the three variables of cost, schedule, and 


performance. 


II. WEAPON SYSTEM PROCUREMENT 
This chapter will investigate cost and schedule growth 
trends in weapon systems procurement over the last few decades 
as well as discuss some of the recurring themes in the cost 
estimating process. It 1S important to understand the 
environment within which cost estimating relationships were 
prepared. Thus, Chapter II sets the stage for further 


discussion of specific cost, schedule, and performance models. 


A. GROWTH TRENDS 

Several studies have been performed which have compared 
rates of cost and schedule growth at different periods in 
time. One of the first was a 1972 General Accounting Office 
(GAO) Report which quantified, on a large scale, the weapon 
system acquisition experience. Schedule slippage was a 
problem, especially for missiles, electronics, and vehicles 
and ordinance systems. Figure 1 shows this schedule slippage 
between the planned Initial Operating Capability (IOC) date 
and the then current estimate of IOC. (GAO, 1972, p. 45) 

While this figure shows growth numbers which cause some 
concern, the fact is that the cost of this schedule growth was 
far less than either the costs of quantity or estimating 


changes. One must also keep in mind that the initial 
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estimates may be at fault and therefore, relative increases in 
schedule may be misleading. 

A 1979 study by Rand Corporation tracked the performance, 
schedule, and cost results for 1970s programs. When result to 
goal ratios were calculated, the means in Table 1 were 
obtained. 

The figures in Table 1 are not weighted by the size of the 
program budget. "When weighted by program cost, the average 
cost-growth ratio for the 31 programs was about 1.14, 
reflecting somewhat lower cost growth in the high-value 


programs." (Dews, et al., 1979, p. 26) 


Reductions in development cost, development schedule, 
production cost and production schedule growth rates have 
generally continued into the 1980s. There is however, 
conflicting data on cost and schedule growth = rates, 
particularly for programs in the 1970s. An Institute for 
Defense Analysis Report provided the rates in Table 2 based on 
42 programs during the 1970s. (Tyson, et al., 1989, p. IV-2) 


Table 1. RESULTS TO GOALS RATIOS 


—— — 5 


SCHEDULE 


Cost 


A cursory look at Tables 1 and 2 will reveal that there 





are significant differences in what appear to be similar 
measures. This demonstrates one of the problems associated 
with statistically based studies. In this case, the results 
are based on different samples, 31 and 42 respectively, but 
the later must contain most, if not all of the former’s 
samples. Even with similar databases, differences in the 
underlying assumptions about the data, treatment of outliers, 
and specific definitions of the variables can yield different 
results. This topic will be revisited later in the thesis 


when specific cost models are examined. 


B. COST ESTIMATION METHODS AND MODELS 

The above discussion of growth trends only hints at the 
various methods and models used to track and predict weapon 
systems costs. Methods include analogy, bottom-up engineering 
estimates, and statistical analysis. While all three are 
means toward the same end, the amount of time and resources 


Table 2. COST AND SCHEDULE GROWTH RATES FOR 1970s PROGRAMS 


DEVELOPMENT COST GROWTH 
DEVELOPMENT SCHEDULE GROWTH 


PRODUCTION COST GROWTH 


PRODUCTION SCHEDULE GROWTH .is 





required of each vary greatly. Analogy may have its basis in 
either or both of the other two methods but it is simply an 
expert’s opinion, based on comparison with other similar 
systems, as to what a new system should cost. Engineering 
estimates require the most time and resources to complete. 
They are detailed estimates at the lowest levels of the work 
breakdown structure, aggregated to form an estimate of the 
total system. They rely on a host of experts and managers at 
all levels of an organization to estimate costs based on a 
proposed design. Statistically based estimates, on the other 


hand, do not necessarily require intimate knowledge of the 
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particular system to be costed. Instead, they seek to explain 
cause-effect type relationships between various parameters and 
some independent variable. Preparation of these models 
require a moderate level of effort, particularly in the data 
gathering phase. The quality of statistically based 
estimating relationships is of course a function of the data 
set employed. This idea will be revisited in later chapters. 

The purpose for which costs are to be estimated largely 
determines the method employed. The focus of this paper will 
be on statistical methods which enable one to evaluate the 
cost of various input alternatives. 

This leads to the question--Inputs at what level? 
Resources required for a weapon system can be categorized in 
many different ways. Models can be created at the system, 
sub-system, or cost element levels of the work breakdown 
structure, by procurement phase, or some combination of both. 
Each successive drop in the work breakdown structure requires 
additional data which may not be comparable or even available. 
Similarly, availability of downstream operating and support 
and disposal costs was limited. Therefore, the studies found 
in the course of research all concentrated on sub-system level 
or higher estimates of the weapon system development and 


production procurement phases. 


a 


C. TECHNOLOGICAL ADVANCE 

One theme which underlies the cost, schedule, and 
performance tradeoff is that of technological or state-of-the- 
art advance. In 1966, the conventional wisdom was that "we 
Simply cannot see a quantitative solution to the problem of 
relating performance characteristics desired from a 
development to state-of-the-art advance."(Glennan, 1966, p. 9) 
As a result, early attempts to place a value on this advance 
were subjective estimates based on expert opinion. However, 
in the early 1970s, one study related technological parameters 
in aircraft turbine engines to the time to pass its 150-hour 
Model Qualification Test. (Alexander and Nelson, 1972) This 
study and further refinements have replaced subjective 
evaluations with objective measurements of technological 
advance. This theme will be evaluated in more detail in the 


following discussion of the various models. 


D. OBJECTIVE 

The underlying objective in weapon system cost estimation 
is that one should be able to predict costs as a function of 
the performance parameters of a system. Ideally, in terms of 
minimizing cost and effort, those parameters should reflect 
the performance of the system as a whole. Unfortunately, the 
performance of a system 1s a function oof design 
characteristics which are inexorably linked with each other, 


and with technological capabilities which impact on schedule. 


eZ 


In the end, only one set of technical specifications is 
selected and the system is delivered on one date. This makes 
comparisons of tradeoffs difficult since one can never know 
for certain what a system with different specifications would 
have cost nor when it would have been delivered. 

The literature linking cost, performance, and schedule is 
by no means abundant. This is due in large part to the sheer 
complexity of the interrelations between performance 
characteristics and technical specifications as well as the 
unique missions of the wide variety of weapon systems. Prior 
large scale tradeoff studies have focused on the aircraft 
industry. In particular, cost estimating relationships have 
been studied for the airframe and turbine engine subsystems. 
One might expect that aircraft would be more widely studied 
than, for example, missile systems. The technology is more 
directly transferable to civilian application, pertinent data 
with respect to costs and specifications are more easily 
accessible, and the industrial base has been more widely 
developed. More recent works on the subject typically refine 


and adjust earlier studies. 


ise, 


2 LT... AIRCRAFT TURBINE ENGINE COST MODELS 
This chapter will investigate the theories behind several 
aircraft engine cost models and the tradeoffs between and 
among the three pillars of cost, schedule, and performance. 
The chapter is organized in three sections, each dealing with 


a specific set of cost or schedule estimating relationships. 


A. TIME VERSUS PERFORMANCE 

This first section on cost models relates the performance 
of aircraft turbine engines to some measure of time. Two 
assumptions are necessary at this point: first, that the 
engine may be described by some combination of performance 
parameters and second, that the marginal increase in 
performance over time displays some historical continuity 
(Alexander and Nelson, 1972, p.3). By definition, these two 
assumptions must hold for essentially all parametric-based 
cost estimating relationships if they are to be effective and 
reliable. 

When seeking to describe the performance characteristics 
of a particular engine, several key factors come to the 
forefront. Among these are engineering considerations such as 
turbine inlet temperature, maximum thrust, weight, fuel 
consumption, speed, and a pressure term. Other 


characteristics which may not reflect performance include such 


14 


dimensions as length and diameter. None of the above 
parameters explicitly brings the technical complexity of the 
system nor time to develop into the equation. It is with this 
foundation that Alexander and Nelson sought to quantify the 
relationship between the "technology index" and some bundle of 
performance parameters and physical characteristics. 
1. The Technology Index 
In Measuring Technological Change: Aircraft Turbine 
Engines, Alexander and Nelson defined the technology index as 
number of calendar quarters (4th quarter 1942 equals one) to 
Byiiieve the 150-hour Model Qualification Test (MQT).” Their 
attempt to relate performance parameters to a length of time 
implicitly attempts to explain the time versus performance 
tradeoff. As a result, input variables can be varied to 
achieve desired results in terms of time to the 150-hour MOT, 
or alternatively, time to MQT can be predicted based on a 
given set of input variables. 
a. Technology Trend Model 
Several methodologies were used to determine the 
best fit model for describing performance versus time. 
Linear, quadratic, semi-logarithmic, and full logarithmic 
forms were tested with the expectation that one would produce 


a significantly better correlation between performance and 


‘The MQT is the final military qualification, after which 
the engine is considered to be sufficiently developed for 
iiseallatwon in a production aircraft. 
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time. Appendix A contains the full set of selected regression 
equations cited by Alexander and Nelson. They include 
equations which incorporate cruise variables, pure performance 
variables, as well as technical parameters. Equation (1) 
below shows the result of the best regression to be in the 
semi-logarithmic form (Alexander and Nelson, 1972, p. 21): 


Tech= -1187.5 +1561n(Temp) +18.81n(Thrust) -26.5l1n(Wgt) 


=20-6in(erc) +t1l1.71n (0) tise Otero) (1) 

Where: 
Tech = Quarters to MQT; (4th quarter 1942 = one). 
Temp = Turbine inlet temperature; Degrees Rankine. 


Thrust = Military sea level static thrust; Lbs. 
Wgt = Engine weight; Lbs. 


SFC = Specific fuel consumption at military sea 
level static thrust; (Lb/hr)/Lb thrust. 


Q = Maximum dynamic pressure; lb/ft%*2. 


Dummy variable; one if turboprop, zero 
otherwise. 


Prop 


The R-Squared value for equation (1) was reported 
as .903 with a standard error of the regression of 9.6. While 
the high R-squared value is impressive, the standard error of 
9.6 quarters (roughly 2.4 years) implies that approximately 
two thirds of future developments could be predicted within a 
five year range. This is by no means satisfactory when a 
project manager is faced with a decision concerning the 
tradeoff between performance and schedule. However, the 


incremental changes in_- schedule (MQT) when altering 
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performance parameters may provide a more acceptable measure 
on which to base tradeoff decisions. 
b. Revised Technology Trend Model 
Nelson refined the earlier time-performance model 
and related it to development and production costs of aircraft 
turbine engines. This later work, entitled Relating 
Technology to Acquisition Costs: Aircraft Turbine Engines, 
made extensive use of the previous data base but some 
revisions require mention. Some of the more important changes 
were: 
¢ 12 turboprop/turboshaft engines were removed because 
detailed development and production costs were not 
available; 


e 4 engines were removed because they failed the 150-hour 
MOT; 


e 5 engines were removed from the military sample and 
included in a commercial sample. (Nelson, 1974, p. 6) 

The resultant data base consisted of twenty-six 

Military turbojet and turbofan engine programs plus eleven 
commercial engine programs ranging from the 1940s through the 
late 1960s. Also of note were two changes made to the 
performance parameters. Maximum thrust was substituted for 
military sea level static thrust and a new pressure term was 


introduced.“ As with the previous study, a stepwise least 


“This term is the product of the maximum dynamic pressure 
of the flight envelope and the pressure ratio of the engine. 


JG 


squares procedure was used to determine the best fit 
equations. Further, the technology index was renamed Time of 
Arrival (TOA) to more accurately reflect that it is indeed a 
measure of time, which substitutes for technology. Equations 
(2) and (3) below are the results of the regression procedure 
and they reflect the TOA for the twenty-six military engines 
(TOA,,) and the thirty-seven military and commercial engine 
(TOA,,) databases, respectively. 
TOA,.= -856.4 +110.111n(TEMP) +11.411n(TOTPRS) (2) 
-26.08ln(WGT) =-16.0ZIn(SFCMIE) = +16" 3 7in (2eeMess) 
R*=.96 SE=6.9 
TOA,,= -—7e#l2 . 8Saet9O8- 151m TEMP) + lee (TOTP RS) (3) 
-26.471n(WGT) -15.671n(SFCMIL) +19.041n(THRMAX) 
+9.86 (MCDUM) 
R*=.96 SE=6.1 
where MCDUM=military-commercial dummy (1l=comn., 
O=mil.) 
The revision to the original data has led to an 
increase in the coefficient of determination, but perhaps just 
as importantly, it has decreased the standard error of the 


estimate. 


“Other variables are as previously defined. 
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B. COST VERSUS PERFORMANCE MODELS 

Watts attempted to relate the cost of aircraft turbine 
engines to several technical and performance parameters in his 
1965 work entitled Aircraft Turbine Engines-Development and 
Procurement Cost. Many of the technical parameters have been 
discussed in the previous sections, however, it is necessary 
at this point to define and discuss the various cost variables 
used in his study. The availability of cost data is perhaps 
the most binding constraint when attempting to quantify a 
cost/performance relationship. There are no hard and fast 
numbers like thrust or temperature which can be easily 
measured (or designed) to calculate costs. Instead, the cost 
estimator is forced to rely on values generated by contractor 
cost accounting systems, supplemented by contract values from 


government procuring agencies. Because of the incomplete data 


availability at the cost element level (i.e., engineering, 
production, tooling, etc.), Watts concerned himself with costs 
at the aggregate level. He divided these costs into two 


categories--development and procurement--the sum of which is 
total acquisition cost. 
1. Development Costs Defined 
It is often difficult to distinguish where development 
ends since designs are frequently subject to engineering 
modifications after one or more lots have been produced. 


Watts treats all product improvements over the life of the 


ing 


system as development costs. Specifically, those production 
costs include: 

.--initial contractor preliminary design, subsequent 
engineering, prototype tooling, material, fabrication, 
assembly and bench testing of scale or full-size 
components or complete engines to and including 
qualification testing to military acceptance 
specifications. Also included is the cost of production 
tooling. Afterburners or nozzles are considered as part 
of the basic engine, as is the reduction gear on a 
turboprop. Sustaining engineering involving factory 
liaison, training, preparation of manuals, etc., are not 
considered as development but as part of the production 
costu. (Watts, 1965 7 ameeec) 

Five dates at which development costs were measured 
were the Preliminary Flight Rating Test (PFRT), the MQT, and 
the times of delivery of 100, 500, and 2000 engines. 

2. Procurement Costs Defined 

Simply defined, the procurement cost was "the total 
cost to fabricate and assemble complete engines including 
labor, materials, overhead and profit." (Watts, 1965, p. 9) 
Adjustments due to inflation were made using the average 
hourly earnings for aircraft engines and parts production 
workers. All adjustments were made to obtain values in 1964 
dollars. 

3. Cost Models 


Multivariable cost models were discarded as either 


unreliable or unreasonable due to the small sample size.’ 


‘The data was divided into samples of eight turbojets and 
Six tumboprops. 
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For that reason, "it was decided to relate development and 
procurement cost to a single technical parameter that would 
describe the size of the engine--the most obvious ones being 
thrust or horsepower." (Watts, 1965, p. 40) However, quantity 
was entered as a second independent variable in an attempt to 
account for the difference in development costs at different 
quantities. 
a. Development Cost Model 

Development costs versus engine size and quantity 

equations are presented below for the turbojet and turboprop 


engine samples. 


Turbojet: Y,. =0.139eqmRe**o°-°"">! (4) 
R*=0. 92 SE=47.25 CV=24.11% 
Turboprop: hae Soeeeeie on n°? “alge °72 4 (5) 
R*=0.95 SE=11.06 CV=16. 99% 
Where: 
Yaev = Cumulative development cost ($million) 
T = maximum thrust (lbs) 
E = equivalent shaft horsepower 


quantity 
b. Procurement Cost Model 
The procurement cost models were separated into 
two turbojet (those with and those without afterburners) and 


the turboprop categories. Results of these models were: 


Zi 


Turbojet with afterburner: 
Yoroc=0 . 18 700TO 247 0°° ae (6) 


R*=0.80 SE=84.78 CV=46. 84% 


Turbojet without afterburner: 


Vorse= 0%. S197 Oe ieeeaOme rea (7) 
R7=0.78 SE=80.47 CV=48. 92% 

TULDOPEOP : 
Yosso=4. B6224 Ros 9 Omen (8) 
R7=0.95 SE=11.91 CV=11.58% 

Where: 


Yproo™ Cumulative average cost/engine ($thousand) 
c. Acquisition Cost Models 
The acquisition cost model is merely the algebraic 
sum of the development model and the procurement cost model 
evaluated at varying quantities. Figure 2 graphically depicts 
this relationship for turboprop engines at acquisition of 100, 
500, and 2000 engines, respectively. 
Large, in his 1970 work entitled Estimating 
Aircraft Turbine Engine Costs, used Watts’ work as a starting 
point for developing his own cost estimates. By 1970, the 
database available for study had grown to twenty-two turbojet 
engines. This made a more statistically reliable model 


possible. 
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Figure 2 


Like Watts, Large related development costs’ at 
five milestones to engine performance parameters. Adjustments 
were made for price-level changes and differences in quantity. 
In particular, the development cost at MOT (Dyo.;) equations he 
found all had thrust as one explanatory variable with either 
speed, altitude, or thrust per unit of outflow as the other. 
The best fit equation is shown below. 


Dag=0'.73-“*A’’”” (9) 


“Production tooling costs were excluded from development 
costs. 
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R7=.95  SE=29 (1969 $M)* Cv=14% 
where: 

F= maximum thrust (thousands of lbs) 

A= absolute altitude (thousands of feet) 

Development costs at other quantities can be 
calculated using the above equation and the development cost 
ratios listed in Table 53. The ratios were based on eight 
engines’ total cost of development through the indicated 


quantity compared to development cost through MOT. 
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Table 3. DEVELOPMENT COST RATIOS 


100 ENGINES 





hun 
7 | [| 


Because extrapolation between the above quantities 


is rather difficult, Large added quantity to the development 
cost estimating relationship. The resultant equation, shown 
below, is: 
D.= 04245 Ae. (1969 $M) (10) 
The decrease in the R’* value to .85 and increase 
in the standard error to $52 million reflect the uncertainty 
introduced by the substitution of a third explanatory 


variable. 


‘To convert 1965 dollars to 1969 dollars, multiply 1965 
dollars by 1.22. This index is based on production workers’ 
average earnings. 
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When production costs were examined, Large states: 
After duly attempting to correlate cost and every 
combination of engine characteristics that appeared 
reasonable or unreasonable, we believe it is fair to say 
that no single or multi-variable estimating equation can 
predict the cost of all engines in the sample with 
acceptable accuracy (Large, 1970, p. 15). 

As a result, the sample was broken down into three 
Classes based on technical design features and material 
composition. Class I engines (three) all had one stage 
compressors and were considered obsolete for military usage. 
All four of the Class III engines had similarly large 
proportions of expensive superalloys and titanium. Class II 
engines contained both afterburning and non-afterburning 
Samples, but in general they had larger turbine inlet 
temperatures than Class I, more common metals as compared to 
Class III, and comparable multi-stage compressors within their 
erass . 

Large used the samples from Class II to determine 
the regression equation which best fit those points. Parallel 
lines were then hand fit to the points in Classes I and III. 
Quantity was introduced as a second explanatory variable 
resulting in the equation for Class II of: 

C,= 2. 61F Q70-154 (11) 

where C,=unit cost in thousands. 

At this point, the R-squared value for Class II 


engines was 0.90, the standard error was $31 thousand, and the 


coefficient of variation was 14%. In order to convert unit 
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cost to cumulative average cost, unit cost was multiplied by 
the quantity (1/b+1). Simplifying further, the total cost of 
development and production (C,) for the three classes can be 


summarized as follows; 


Class I: C.=.24F 7A ?'O"* +22 096R “seme: (12) 
C,=27.8F°°M:®9°' + .096F-%Q°%5 (13) 
Class II: Cy=. 24F Al 70 + oar cons (14) 
Cr=27.. OE 2M, °°O (ier Ons a) 
Classy iis) C,=.24r" AO + soe Cae, (16) 
€5=27 | BE 2Mt 20 ae So orcs Oaune (17) 


where M = maximum speed in Mach number and other variables are 
as previously defined. The additional developmental cost 
equation, incorporated into the total cost equation, provides 
the estimator with two methods to better estimate a range of 
likely engine costs. 

Up until this point, we have presented different 
models of performance versus time and cost versus performance 
relationships for aircraft turbine engines. What remains now 
is to link the three characteristics together so that 
tradeoffs between cost, performance, and schedule may be 


comparatively measured. 


C. COST, SCHEDULE, AND PERFORMANCE MODELS 


Nelson, in Relating Technology to Acquisition Costs. 
Aircraft Turbine Engines validated the variables that Large 


found to be important predictors of cost. However, the 
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addition of the time-of-arrival variable eliminated the need 
to separate engines into different classes. In effect, this 
eliminated one source of variability from the total cost 
equation. As with the previous cost studies, two adjustments 
were made to the data--one for price level changes" and the 
other for quantity and engine configuration (homogeneous 
models). 

Three different cost models were presented. They included 
a standard variables model, a time-of-arrival model , anda 
technology parameters model. The standard variables equation 
included explanatory variables which were proven to be 
important in past studies plus development time and/or 
quantity. The time-of-arrival equation includes the standard 


* 


variables plus TOA and/or AToa*™*. The technology parameters 
equation included a mixture of explanatory variables present 
in the other two models. 

Development cost equations were prepared for two 
milestones--those costs up to MQT and total development cost. 
Development cost to MQT included design, engineering, tooling, 
materials, fabrication, and assembly and testing of components 
and complete engines. They also include overhead and profit 


since the equations predict total cost to the government. 

Similarly, production costs were total cost to the government 
“See Appendix B for price level adjustment factors. 
““ATOA is the difference between actual and predicted 


dates of arrival at MOT. 
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through Cer antec. Nelson used two methods to estimate 
production costs. The first method involved an estimate of 
the 1000th unit cost and an estimate of the slope of the 
cumulative average production unit progress curve. The second 


and more straightforward method predicted cost as a function 


of quantity plus other explanatory variables. The former 
method will be omitted because of its complexity. The 
procedures do yield similarly significant results. Nelson 
states: 


As an overall measure of comparability between these two 
approaches, the TOA equation for unit 1000 cost and the 
technology parameters equation for slope were used to 
predict the costs of the 88 input observations for the 
cumulative cost regressions. The calculated coefficient 
of determination is 0.968, which compares favorably with 
the R’? for the TOA cumulative production cost equation 
(0.978). (Nelson, 1974, p. 49) 

1. Development Cost Models 

Below are the three development-cost-to-MQT equations 
presented by Nelson. 
Time-of-Arrival Model: 
In Dygp=-1.4461+.08538T,+.496301n THRMAX (18) 
+.04099ATOA,,+.413681nMACH 
R*=0.961 SE=0.182 (+20%, -1%) 
Technology Parameters Model: 

inwDysp= tei? 23+.0711L OS ei Z 02 in Poe rine. (19) 

+.58532 1n MACH -.26470 l1n WGT 


R7=0.882 SE=0.317 (+37%, -27%) 


Z8 


Standard Model: 
my Dent sto? 7 9+0. 07463T,+.47611, In THRMAX (20) 
+.5112 1n MACH 
R?=0.858 SE=0.329 (+39%, -28%) 
where Dy, — development cost to MOT (1973 $M) 
Tp = development time from start to MQT (quarters) 
MACH = maximum flight envelope Mach number 
THRMAX = maximum thrust, sea level static (1bs) 

By adding quantity into the stepwise regression 
procedure, Nelson found total development costs (TDS) as 
follows: 

Time-of-Arrival Model: 
im TDSs= 0.23%984+1.0193 ln MACH +0.06228 ln QO (21) 
+.442511n THRMAX +0.01418 TOTDEVTIME 
R7=0. 927 SE=0.209 (423%, -19%) 
Technology Parameters Model: 
Piet Do—-10 48 5uu.0098 Im MACH +0.07119 1in Q (22) 
Oma a0o neawGd +1. 5642 in TEMP 
R*=0. 930 SE=0.204 (+23%, -19%) 
Standard Model: 
Pith o—O.7 9747 +1.2867 dan MACH +0.08146 1n Q (23) 
+0.398841n THRMAX 


R7=0.858 SE=0.329 (+39%, -28%) 


Zo 


2. Procurement Cost Models 

Cumulative production costs were based on 88 data 
points from 18 turbojet and turbofan engines. Several items 
should be noted regarding these curves and the statistical 
evidence supporting them. First, since the data points are 
Cumulative measures of costs and quantities, there is a 
potential problem with multicollinearity. As a result, errors 
in the data points can not be assumed to be independent--the 
effect being that the standard errors are suspect. Second, 
engines with more data points will have a stronger effect on 
the outcome of the regression line coefficients. The first 
three production cost equations contain a dummy variable for 
engine manufacturer--its value being one for Pratt & Whitney 
and zero for all others. No explanation is given for the 
difference in production costs between contractors. The four 
procurement cost models do have R* values greater than or 
equal to 0.948 so there is the potential to estimate a 
reliable range of values for procurement costs. The four 


models in order of their R* values are: 


Time-of-Arrival Model with Manufacturer Dummy: 
ln PROCS = -7.9854+0.92753 Im @ +0.77831 Im THRMAX FZ 
+0.01542ATOA,,, + MFRDUM +0.37779 1n MACH 


R*=0.978 SE=0.198 (+22%, -18%) 
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Technology Parameters Model with Manufacturer Dummy: 
ln PROCS = -25.130+0.86943 1n Q +0.86883 ln THRMAX (25) 
+ MFRDUM +0.30170 In MACH +2.2107 ln TEMP 
R’7=0.968 SE=0.235 (+27%, -21%) 
Standard Model with Manufacturer Dummy: 
ln PROCS = -8.5461+0.87079 1ln Q +0.90865 ln THRMAX (26) 
+ MFRDUM +0.24242 1n MACH 
R’=0.963 SE=0.251 (+29%, -22%) 
Standard Model without Manufacturer Dummy: 
In PROCS = -7.9417+0.84172 1n Q +0.84755 1n THRMAX (26) 
+ 0.21462 1n MACH 
R*=0. 948 SE=0.295 (+34%, -26%) 
where PROCS = procurement costs through Q engines (1973 $M) 


ATOA,,., = time-of-arrival index of last growth engine 
(calendar quarters) 


It should be stressed that the TOA variable, being a 
function of temperature, pressure, thrust, weight, and fuel 
consumption is confounded with THRMAX in equation (24). 
Similarly, any measure of time must also be confounded to some 
extent with other variables in the equation. That is to say, 
one can not say for sure what the real effects of time are on 
the dependent variable. This is a possible explanation of why 
the development time variable is linearly related to 
development cost when we would intuitively expect it to be 


quadratic in form. A second interpretation of this linear 


Suk 


relationship is that it approximates the upward sloping right 
half of the quadratic. This effect does not undermine the 

values of the coefficients or the forms of the variables, but 
it does affect the reliability and precision of the standard 


error term. 
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Iv. AIRCRAFT/AIRFRAME COST ESTIMATING MODELS 


Initial aircraft cost models focused on production and 
generally related direct labor hours per pound to production 
quantity. These plots resemble the familiar learning curve 
and were of the form y=Ax”. Over a period of years, the "A" 
value increased significantly for fighter aircraft indicating 
that it was taking more direct labor hours per pound to 
complete an aircraft. When further analyzed, speed of the 


aircraft was indicated as a possible explanatory variable. 


A. COST VERSUS PERFORMANCE 

Virtually all of the airframe cost models have both weight 
and speed as explanatory variables. While speed is most 
definitely a performance variable, weight seems more like a 
technical variable. However, weight and particularly ratios 
involving weight (i.e., thrust/weight) can also describe the 
performance characteristics desired of an aircraft and 
therefore it should be considered as one possible measure of 
system performance. 

Glennan found the following relationships” using weight 


and speed as explanatory variables (Glennan, 1966, p. 8): 


*The database consisted of the F-84, F-86, F-86D, F-89, 
F-100, F-101, F-104, F-105, F-106, B-47, B-52, B-58, and F-4. 


Se 


log Ez; = .41°+7383 Pogqew +i so? 69-5 (27) 
HOGeE,5, = -46er. Sim log Ww +1. 34a loa. (28) 
log C,, = .72 41.04 lLogew +ieeA2 Teams (29) 


where E,, = Cost of engineering including flight test through 
25 aircraft (1962 SM) 


E100 = Cost of engineering including flight test through 
100 aircraft (1962 $M) 


C,, = Cost of airframe program through 25 aircraft 
(1962 $M) 


W 


AMPR weight of aircraft (thousands of lbs) 
S = Maximum speed of aircraft (Mach) 

The R* values for equations (27), (28), and (29) are .92, 
-94, and .97, respectively. However, the large range of the 
dependent variable produces deviations in the 20 percent 
range. Notice that the dependent variable E is an attempt to 
break down costs by functional cost element. This is a result 
of the databases available at the time. Also notice that the 
accuracy (R*) of the total cost is better than the engineering 


cost element. Large states; 


The accuracy of the overall estimate is always better, 
however, than the accuracy of the individual elements, 
because the data are inconsistent at the cost-element 
level. What one company calls engineering, another 
company calls tooling, or a given company will change 
definitions to conform to cost accounting standards, and 
it has never been possible to adjust the data to eliminate 
all discrepancies. [Therefore,] at the highest level-- 
aircraft cost--comparisons are most valid. (Large, 1981, 
p. 18) 


More recent total cost/performance relationships involving 
weight and speed are presented below. The first, equation 


(30), represents a 1976 study which contained 24 airframes. 
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The second airframe relationship, equation (31), relies on a 
sample of 13 post-1960 airframes which is considered to be a 


better predictor of future aircraft. (Hess and Romanoff, 1987, 


p. 45) 
PROG, = 6.22 AUW'’*® sp:??? (30) 
R? =.88 Shia i207 
PROG wSe2i57_ AuWi °° uSP.73° (31) 
R* =.85 SE = .36 
where PROG,,., = Cumulative total program cost for 100 


aircraft (1977 S$thousand) 
AUW = Airframe unit weight (1b) 


SP = Maximum speed (knots) 


B. TIME VERSUS PERFORMANCE 

The search for a technology index for aircraft roughly 
paralleled that of the time-of-arrival method for turbine 
engines. The natural equivalent to TOA was first flight date. 
Stanley and Miller sought to quantify this technological 
change in fighter aircraft in their 1979 work entitled 
Measuring Technological Change in Jet Fighter Aircraft. All 
combinations of linear and logarithmic variables were tested. 
They determined that "the log-linear equation form 
unambiguously characterized the growth in U.S. fighter 
technology better than the other equation forms." (Stanley and 
Miller, 1979, p. 21) They also performed several regressions 


to determine from which point in time the first flight date 


oS 


would be measured. Ultimately, and after careful analysis, 
January 1, 1940 was selected as the date from which first 
flights would be measured. The database consisted of 25 new 
designs with first flights ranging from 1946 to 1976. Six 
technology equations were presented with R* values ranging 
from .945 to .882 and standard errors ranging from ~2i7eeo 
-160. The first equation only is presented below: 

In (t) = 3.878 + .065[ (Thrust) (V,,,) /100W,,] (32) 

+.406[BR/1000] +1.409[SLF/10] 


+.939[PF] -.093[carrier capability]’ 


where t = months since 1 JAN 1940 
Wat = Combat weight (1b) 
BR = Breguet range” 
SLF = Sustained load factor” 


PF = Payload fraction (Max gross weight-full internal 
weight divided by maximum gross weight) 


The remaining five equations all contain the thrust and 
velocity variables, either Breguet range or Breguet range 
factor as well as some combination of payload fraction, 
internal or total fuel fraction, sustained load factor, and 


carrier capability. 


"1 denotes no capability, 0 denotes capability. 


“The range calculated using the Breguet range equation 
assuming all internal fuel is used at cruise conditions and 
the aircraft is initially loaded to its maximum gross weight. 
“The maximum load factor the aircraft can sustain in 
level flight at an altitude of 25,000 feet and at a Mach 
number of .8 at its combat weight. 
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It is evident from the above definitions that both 
performance and technical characteristics must be known before 
equation (32) can be used. This equation truly displays the 
paradox in cost estimating relationships. Namely, attempts to 
locate better predictors will invariably make the equation so 
cumbersome that its use as a simple estimating tool becomes 
futile. Nevertheless, these somewhat obscure variables all 
seek to describe some measure of output desired from an 


arrcraft. 


C. COST, SCHEDULE, AND PERFORMANCE MODELS 

Only one of the following models explicitly includes some 
measure of development time as an explanatory variable for 
Cost. However, the inclusion of a measure of time, in the 
form of a technology index, does provide some foundation with 
which one can attempt to trade off certain design parameters 
or performance characteristics with cost and schedule. 

Glennan was among the first to include some measure of 
state-of-the-art advance and performance characteristics to 
airframe costs. He added another variable "A" to equation 
(29) which was a subjective measure of state-of-the-art 
advance obtained by polling experts in the field.” He states; 

Their rankings showed a high degree of consistency which 

could be considered as giving us greater confidence in the 


measure than if there had been little correlation between 
their responses. On the other hand it might simply 


“The variable "A" ranged from zero to twelve. 
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indicate that they had all formed opinions based upon the 
difficulty the developers had with the development. 
(Glennan, 1966, p. 9) 


The resultant equation for cost through 25 airframes is; 
log C,,. = .89 + 1.07log W +1.45 log S -.25 log A(33) 

R* = .97 
Glennan sought to cbtain a more objective measure of 
technology and he subsequently replaced "A" with the variable 
"T" which he defined as the number of months from January 1944 
to the first flight date of the aircraft. This equation is 


shown below; 


log Cj, = 2.12") 96logun Ff. 94) loge sd.50 le uml (34) 
R* = .98 
Glennan noted that the coefficient for "T" "appears to 


have drawn some of its statistical significance from other 
factors which were changing over time and which tended to 
drive development costs upward." (Glennan, 1966, p. 12) This 
relates to the problem of confounding as described earlier. 
In the only attempt to relate development time (D)*” and 
performance to cost, Glennan found that the coefficient of 
development time was never significant. This suggests, at 


least based on the 13 pre-1955 aircraft samples, that changes 


*Recall that C,, is measured in 1962 dollars. 


“Arbitrarily defined as time in months from letter 
contract to delivery of 25 aircraft. 
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in development time do not affect costs significantly. The 

equation for comparison purposes is; 
Heqgme . =o tt tOlog W tlv46 log S -.15 log D (35) 

R? = 297 

In 1973, a J.W. Noah model was prepared for the Navy 
(Large, 1981, p. 18). It separated costs into two categories- 
-recurring and non-recurring. The model contained two 
subjective measures. The first, a technology index, was 
essentially a time based number which rated all fighter 
aircraft through the F-14. The second was a dummy variable 
which held the value of zero or one depending on the 
complexity of the airframe design. The remaining explanatory 
variables were speed and some measure of weight. The Noah 


model equations are; 


be 
] 


rome oe. O0Cose et OSS 8T a1. A071R +6.74920 (36) 


Q 
N 
li 


MOS. 05 +. 11 Soca 2064Ts-10248A +97.631d (37) 
where C, = Non-recurring $/lb of airframe weight (thousands) 


C, = Recurring $/lb of airframe weight (Cumulative 
average cost at 100 airframes) 


S = Maximum speed at altitude (knots) 


T = Technical index 

R = Gross takeoff weight divided by airframe weight 
A = Airframe weight 

d = Complexity dummy (0 or 1) 


These equations do not stand alone when one has to 


estimate the total cost of procurement. Since quantity is not 


Sy, 


explicitly defined, some characteristics of the production 
learning curve must be known (or estimated) to predict total 
cost. While this may not be as elegant as an equation using 
quantity, it does provide the estimator with a second method 
with which to evaluate program costs. 

The most recent work on airframe cost estimating 
relationships by Hess and Romanoff was published in 1987. 
Their recommended cost model for program costs was previously 
presented as equation (31). However, they did produce 
equations which incorporated a time variable. It is important 
to highlight these results because they show how the form of 
a cost estimating relationship can affect the outcome of a 
cost estimate. Two equations are presented below--the first 
with a linear incorporation of first flight date (FFD) and the 
second with a logarithmic incorporation of FFD. 

Linear Incorporation of FFD: 

PROG, = 2.82 EBW'°? Sp-&9 | :00240(RRD) (38) 
R* =.89 SE =" .27 
Logarithmic Incorporation of FFD: 

PROG, 99 = -647 EW’*? SP-°* FD’??? (39) 
R? =.89 SE = .27 


where PROG,.,. = Cumulative total program cost for 100 
aircraft (1977 S$thousand) 


EW = Empty weight (1b) 
SP = Maximum speed (knots) 


FFD = months since 1 January 1940 
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Note that the R’ value and the standard error of the 





regression are identical for both forms. However, these only 
explain the variance within the database. Figure 3 shows the 
effect when points outside the range of the database are 
estimated based on these equations.” The linear incorporation 


of FFD implies an acceleration of costs over time while with 
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Figure 3 


“The curves in Figure 3 are actually based on the sum of 

engineering, tooling, labor, material, development support, 

flight test, and quality control cost equations with the 
indicated incorporation of FFD. 
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the logarithmic transformation of FFD, costs remain nearly 
proportional to time. 

When an aircraft with an estimated FFD of 1995 is 
considered, the linear incorporation of FFD produces an 
estimate of program costs approximately 50 percent greater 
than the equation with the logarithmic transformation. The 
large variation in estimates for those aircraft with projected 
first flight dates outside the range of data is a problem 
common to all equations which include a measure of time as an 
explanatory variable. This is of course true for any cost 
estimating relationship. Any number of equation forms may 
adequately represent a set of points within their range. 
However, when those equations are used to predict values far 
outside the range of available data, the level of uncertainty 
increases. Even though we must rely on the assumption of 
continuity, there must also be some subjective notion on the 
likelihood of expected outcomes. Hess and Romanoff were 
"unable to say which of the two FFD forms [would] more 
accurately reflect future industry experience" and thus 
recommend that equations incorporating the FFD variable not be 
used. (Hess and Romanoff, 1987, p. 50) 

One way to reduce this difference is to update the 
regression equations after some period of time. Certainly, as 
new aircraft are introduced, the equations can and should be 


recalculated. 
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V. TRADEOFFS--PROBLEMS OR PROGRESS 

This thesis has attempted to compile and explain the best 
models which relate performance characteristics, cost, and 
schedule in the acquisition of weapon systems. The theory 
that these variables are interrelated is well documented; 
however, quantitative solutions involving all three are rare. 
There are numerous reasons why this situation exists, however, 
some progress has been made. The following sections describe 
my assessment of some of the problems with quantifying 


tradeoffs as well as some of the progress made in the area. 


A. PROBLEMS 

There are several fundamental problems with trying to 
quantify the tradeoff between schedule and either cost or 
performance. The following list is not intended to be 
comprehensive, but rather, it highlights those problems which 
have a major impact on the accuracy and applicability of 


tradeoff models. Among some of the problems are: 


* cost accounts and allocations 

e database integrity and homogeneity 
e schedule networks 

e funding instability 

°* risk 


°° baselines 
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Cost accounts and allocations and database integrity and 
homogeneity have been touched on in the previous chapters. 
The remaining four problems are common threads which run 
through the weapon acquisition cost estimating process. The 
effect of each is to add a measure of uncertainty and variance 
to the data which can never fully be factored out. Each of 
these items will now be covered in more detail. 

1. Cost Accounts and Allocation 

The desire and, in fact, need in cost estimating is to 
compare apples with apples. While the cost accounting systems 
imposed by the government on contractors go a long way towards 
standardization, there is still substantial latitude in cost 
allocation schemes. This makes inter-contractor comparison of 
costs at the detailed levels more cumbersome because 
differences between accounting systems can never fully be 
factored out. The contractor "dummy" variables in several 
equations support this idea. As we have already seen, this 
results in cost estimating relationships which are generally 
more accurate at the total cost level. 

2. Database Integrity and Homogeneity 

In Simple terms, one could say that database 
integrity” and homogeneity are inversely related to 


uncertainty. One can regress a non-homogenous sample to 


“Integrity in this context is used to mean the degree to 
which the database is suitable for accurately predicting costs 
of future systems. 
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determine a relationship between two or more variables which 
May potentially be very accurate for predictive purposes. 
However, an estimate based on a non-homogenous sample must 
eiisomcarry witheidt aeshigh level of uncertainty. Similarly, 
one could regress homogenous sub-samples and be just as 
uncertain about a prediction because of a small number of 
Sample points. This is the problem with the samples described 
in the previous chapters. The trend has been to incorporate 
some variable (technology) which describes differences between 
the sub-samples and allows the full database to be included. 
Because of the use of non-homogenous samples the general goal 
for accuracy has been approximately plus or minus 20 percent. 
(barge, 1981, p. 28) 

Creating and maintaining database homogeneity is 
perhaps the most difficult and time consuming process for cost 
estimators. Factors such as where research and development 
ends and where production begins is a matter of considerable 
judgement. The MQT date for aircraft engines and first flight 
date for aircraft seem to be the least ambiguous estimates of 
this point. Costs of producing an engine or aircraft to meet 
their respective requirements may logically fall into the 
research and development phase. However, it may be just as 
reasonable to allocate these to production costs. Thus some 
judgement is required to keep these costs comparable among 


contractors. 
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The recent requirements for minimizing life cycle 
costs puts an additional subjective measure on cost 
estimating. Certainly pre-planned product improvements (P°I) 
may add substantial costs to the initial versions of a weapon 
system, but in the long run they reduce the cost of what may 
have been expensive modifications. The question then becomes 
how can the cost of research and development of the P*I be 
properly allocated between the initial system and its later 
improved versions? Attempting to quantify these short term 
costs which have their basis in long term designs adds another 
measure of variance to the total cost equation. 

3. Schedule Networks 

The development of any weapon system is comprised of 
a network of precedence related elements or activities. The 
duration of each element is a function of resources such as 
labor, materials, equipment, and money. By definition, one or 
more of these paths through the elements must be critical. 
That is to say that any delay in one of these critical 
elements, will cause a delay in the overall project. 
Similarly, any reduction in the duration of a critical element 
may cause a shortening of the overall project. 

There are normally three durations when describing a 
projyect--what the government thinks (or requires) it should 
be, what the contractor believes it will take, and finally, 


what the negotiated agreement states it will be. Along with 
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the negotiated duration, there is also an associated cost. If 
the negotiated agreement is longer than what the contractor 
believes it should take, there is an implied slack in the 
Semeractor’s schedule. Theoretically, if this is the case, 
schedule compression imposed on the contractor would have no 
real effect on the price to the government. Even if the 
schedule is compressed beyond the point that the contractor 
believes the project should take, he still may have latitude, 
without increasing costs, in reassigning resources (manpower, 
equipment, etc.) to those activities on the critical path. 
However, one must keep in mind that there are opportunity 
Sests for the contractor. The resources diverted to one 
project may adversely impact upon several others in terms of 
fost, itime, or both. Schedule stretchout also implies 
Sepertunity costs for the contractor. Idle equipment, 
inefficient utilization of manpower, and cost of money are 
very real effects of stretching out a project. Te 1S 
precisely the difficulty in analyzing and quantifying those 
opportunity costs for one, let alone many contractors, which 
makes estimating a cost-schedule tradeoff function so 
gautLicult. 
4. Funding Instability 

Funding instability is perhaps the most nebulous in 

its impact on system acquisitions. One Rand report states: 


No major acquisition program can be planned and managed 
with high efficiency if it faces frequent’ and 
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unpredictable changes in year-by-year program funding and 
production scheduling, even if total program funding 
eventually reaches the originally planned = amount. 
Schedule slippage and cost growth are the closely related 
and mutually reinforcing effects of program funding 
instability. (Dews, et al., 1979, p.. xi) 

The effect of unpredictable funding is to increase the 
risk to the contractor that planned purchases will not occur 
or that the program may be canceled entirely. This short term 
budget focus makes long term efficient and effective planning 
difficult, if not impossible. Cost estimating relationships 
have not been able to take into account the effect uncertain 
budgets have on schedule slippage and cost growth. 

5. Risk 

The idea of risk management in defense acquisition is 
not a new one. Both performance risk and schedule risk have 
cost-measurable implications. Therefore, risk-oriented 
tradeoffs are both possible and desirable from program 
management perspective. 

Many developments are on the cutting edge of 
technology and as such, it would be unfair to place all the 
risk of such a development solely on a-e contractor. 
Recognizing this, the acquisition community has adopted 
several types of cost sharing or incentive type contracts. 
Thus, the selection of a particular contract type influences 
the amount a weapon system will cost. Without specific 


information regarding the type of contract used, and its 


attendant risk, the cost of that risk ultimately gets hidden 
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in some other variable in the cost equation. When data on two 
otherwise similar weapon systems are compared, the use of 
different contract types implies different costs, and by 
extension, different tradeoff possibilities. 
6. Baselines 

The Concept, Development, and Production Baselines, 
approved at Milestones I, II, and III, respectively, set out 
key objectives for cost, schedule, and performance parameters 
(PODINST 5000.2, 1991, p. 11-A-1). These objectives, within 
certain tradeoff limits (thresholds) must be met in order to 
reach the next Milestone. Cost estimating relationships which 
seek to quantify the tradeoff relationships in either 
development or production can not hope to describe the many 
different sets of tradeoff options available during each 
phase. Data is not sufficiently available at these levels to 
quantify a tradeoff relationship for each phase. Instead, the 
data must be aggregated across Milestones--the effect being 
that there is an additional measure of uncertainty when a 
tradeoff equation is applied at one particular Milestone or 
during one particular phase. To put it another way, the 
tradeoff options during the Demonstration and Validation Phase 
and the Engineering and Manufacturing Development Phase are 
quite different. One equation applied to both must 


necessarily be a compromise between the two. 
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A second point regarding the use of baselines is 
highlighted by the growth models presented in Chapter II. One 
thing that must be kept in mind, particularly when referring 
to schedule or cost growth, is the original estimate. Many 
Studies which attempt to show trends in cost or schedule 
growth use ratios of actual to estimated. The problem here is 
quite obvious: which one is really being measured, the quality 
of the estimate or the trend in actual data? Does a high 
growth rate reflect spiraling costs or just a bad estimate? 
An overly conservative estimate would, other things being 
equal, yield a lower growth rate. Two things compound this 
problem. First, estimates at completion are constantly 
updated, and second, different organizations use different 
bases. It is often difficult to discern which (basewe: 
estimate is being used and therefore it is difficult to make 


comparisons between estimates. 


B. PROGRESS 

Given all of the above mentioned problems, the question 
remains as to whether or not any progress has been made in 
quantifying the cost-schedule-performance tradeoff surface. 
Certainly, cost-performance models have been improved over the 
years but this is largely a function of improvements in 
program data tracking and a larger database. Cost-schedule 
and schedule-performance models have not, unfortunately, 


evolved to a point where their estimating effects are 
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Significant. The inclusion of a technology parameter in the 
form of time is a first step in quantifying a cost-schedule- 
performance relationship. 

In this research, the author did not set out to exclude 
any types of systems from consideration. In fact, any model 
which attempted to introduce schedule (time) into a cost- 
performance relationship would have been compiled and 
explained, whether or not it described a weapon system. 
However, this research was quickly focused on aircraft engines 
and airframes simply because these were the only two which 
included some measure of time. 

The possibility that airframe cost estimating 
relationships will improve in the near future is severely 
limited. There is a hope that new variables will somehow 
improve the existing estimates but these new variables will 
most certainly be more complex than the parameters of weight 
and speed. Thus, there appears to be no simple way, given the 
data available today, to quantify accurately and reliably the 


interwoven effects of schedule, cost, and performance. 
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VI. CONCLUSION 

"One of the most difficult aspects of planning military 
procurement would be overcome if a satisfactory method could 
be found for estimating costs incurred by increased 
specifications." (Arrow, 1950, p. 2) No one since has stated 
the problem any more clearly. Many have attempted to quantify 
the cost of increased specifications. The literature is 
abundant with cost/performance models for items ranging from 
reconnaissance drones to jet fighters. Relatively new to the 
end user, statistical packages enable anyone to "invent" cost 
estimating relationships for their particular area of 
interest. The purpose of this thesis was not to review and 
describe the myriad of cost relationships for each type of 
platform. Instead, its primary focus was to compile and 
explain those models which incorporated schedule along with 
cost and performance. A careful review of the models 
described in this thesis will show that only five’ equations 
out of all those presented explicitly had a measure of 
development time as an explanatory variable. Many others, 
however, incorporated a time-based measure of the technology. 
Either approach allows tradeoffs*" to be made between 

“Equations (18) through (21) and equation (35). Note 
that in equation (35), development time was found not to be 
significant as an explainer of cost. 


“Within the respective range of accuracy. 


a2 


schedule, cost, and performance. The wide ranges of data or 
the small sample sizes do not lend themselves to the levels of 
significance required for use by a program manager for 
possible tradeoffs on specific weapon systems. These 
equations are sufficient for long range planning and possible 


ranking of alternative designs with respect to expected costs. 


A. FUTURE RESEARCH 

As discussed in Chapter V, it is the opportunity cost of 
the unchosen alternative that is so difficult to quantify. A 
possible method for estimating these costs could be obtained 
by analyzing contractual changes (modifications) due to 
performance changes or schedule changes. It is well known 
that there are typically many modifications to each contract. 
Some involve schedule extensions while others’ modify 
performance parameters. It is possible that a relationship 
may exist between the dollar amounts of modifications and the 
reason for that modification. Take, for example, a government 
caused delay which requires a modification to the contract. 
There is some negotiated cost that is ultimately assigned to 
that delay. Over a full range of contracts and contractors, 
there may be a sufficient database to develop a cost per time 
estimate. Data gathering may be quite a task, but agencies 
such as the Defense Contract Audit Agency may be able to 


provide some assistance. 
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The studies reviewed in the course of this thesis all 
revolve around research and development and production costs 
and schedules. Operating and support and disposal costs have 
Since become more available. It now may be possible to 
incorporate these costs into equations to quantify procurement 


tradeoffs to minimize weapon system life-cycle costs. 
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APPENDIX A 
This appendix contains selected regression equations which 
include a measure of technological trend as presented by 


Alexander and Nelson. (Alexander and Nelson, 1972, p. 21) 


Best Equation (semi-logarithmic) 
tecn= —lit¢g.5 +i56in(Temp) +18.8in(Thrust) -26.51n(Wgt) 
ero (ore) til. 71m (O) Fils .0 (Prop) (1) 
R*=.903 SE=9.6 
Linear 
Tech= -77.1 + .077(Temp) +.00066(Thrust) -.006 (Wgt) 
-34.4(SFC) +.0094(Q) +1.77 (Prop) (2) 
R*=.832 SE=12.6 
Cruise variables 
Tech= -1501.8 +213ln(Temp) +.861n(Thrust*) -8.41n(Wgt) 
-27.6n(SFC") -3.101n(Q*°) -24.3 (Prop) (3) 
R*=.835 SE=12.5 
Date of first flight 
Flight= -1245.6 +163.71n(Temp) +20.71n(Thrust) 
omc) e=Z20yo ba ShC) +tli,gin(Q) +11.8(Prop) (4) 
R*=.891 SE=10.7 
Pure performance 
Tech= -38.9 +33.4 In(Thrust) -39.0l1n(Wgt) 


-38.91n(SFC) +13.91n (Q)+16.5 (Prop) (5) 


2 


R*=.691 SE=16.9 


Technical 


Tech= -1121.9 +152.91n(Temp) +18.61n(Pressure) 


-7.i7 in. (Airflow) +£6.20 “(ean (6) 


R7=.831 SE=12.4 


Where: 
Airflow= 
Fan = 
Flight = 
Tech = 
Temp = 
Thrust = 
Wgt = 


SEC = 


Q = 


12 15 (8h @) = 


Pressure 


* — 


Total airflow through engine (1b/sec) 

Dummy variable; 1 if turbofan, else 0. 

Date of first flight (4th quarter 1942 =1) 
Quarters to MQT; (4th quarter 1942 = one). 
Turbine inlet temperature; Degrees Rankine. 
Military sea level static thrust; Lbs. 
Engine weight; Lbs. 


Specific fuel consumption at military sea 
level static thrust; (Lb/hr)/Lb thrust. 


Maximum dynamic pressure; l1b/ft%*2. 


Dummy variable; one if turboprop, zero 
otherwise. 


= Overall pressure ratio. 


indicated parameter at cruise speed 
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APPENDIX B - PRICE INDICES 


PRICE LEVEL ADJUSTMENT INDEX* FOR AIRCRAFT ENGINES 


YEAR INDEX YEAR INDEX 
1946 3.824 1960 Diggs) 2 
1947 3.028 Hee 6 ilar) 
1948 Sig Ales, 19,62 1.718 
1949 ie lave, ISN: ero 2 
o> 0 sO 12 1964 LesiG Aes 
Pg St: 2a 03 To 65 ce stat 
SZ fae ae Ml | 1966 12206 
oS 2.5.6 ISN SF 1. 4162 
1954 22439 MENS 3} Ue 5 TMe 
oS 2540 Wwe 9 ee oe 
Sans) Lge e 137 0 ss 3.0) 
LoS |, 2.128 Po 1 1.147 
1958 OZ IS) 9) 4 1.064 
iL SIS ye, ae op) 1973 12000 


* 


This index is based on Bureau of Labor Statistics data 
for average hourly earnings of production workers in the 
aircraft engine industry. 
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APPENDIX C —- ANNOTATED BIBLIOGRAPHY 
This annotated bibliography attempts to provide references 
in addition to those cited in the body of the thesis which 
have either theoretically or quantitatively described the 
relationship between or among the three variables of cost, 


schedule, and performance. 


A. THEORETICAL WORKS AND GROWTH MODELS 
1. Peck, M. J., and Scherer, F. M., The Weapons 
Acquisition Process: An Economic Analysis, The 
President and Fellows of Harvard College, 1962. 
This 736 page book describes the overall nature of the 
weapon system acquisition process. In particular, jee 
describes the theory behind the production possibilities 


curves as a function of input resources and the conceptual 


foundations of tradeoffs between and among those resources. 


2. Zschau, E. V. W., Project Modelling: A Technique For 
Estimating Time—-Cost—Performance Trade-offs in System 


Development Projects (RM-5304-PR), The Rand 
Corporation, Santa Monica, Ca., July 1969. 


Zschau outlines a methodology for estimating time- 
cost-performance tradeoffs. The theoretical model he proposes 
is an extension of the common linear programming approach; 
namely, that each set of desired system output parameters can 
be described as some function of the vast combination of input 
variables and constraints. Problems associated with the 


interdependence of the input variables and the nonlinearity of 
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equations have limited the applicability of this modelling 


approach. 


3. Payne, I. S., Investigation of the Short Range Cost 


Impact of Program Stretchout, Graduate Paper, Air 
Force Institute of Technology, Wright-Patterson AFB, 


OH, September 1975. 


Payne examines the impact of program schedule on cost 


from the corporate financial perspective. The effects of 
program stretchout on several cost categories are 
investigated. Those categories include fixed, direct, 


indirect, overhead, and opportunity costs as well as the 
effect of reduced overhead allocation bases. 
4. Asher, N. J., and Maggelet, T. F., On Estimating the 

Cost Growth of Weapon Systems (P-1494), Institute for 

Defense Analysis, Alexandria, Va., June 1980. 

This paper documents schedule and cost growth in then 
current major weapon system acquisition programs which had 
attained initial operating capability. Note: Two other 
studies on this subject were cited in Chapter II. 

5. Glover, W. L., and Lenz, J. 0., Cost Growth Model for 


Weapons System Development Programs, Graduate Paper, 
Air Force Institute of Technology, Wright-Patterson 


AFB, OH, August 1974. 

In this work, a model is presented which attempts to 
express final development costs as a ratio of initial cost 
estimates to program entropy. Here, entropy is defined as a 
measure of uncertainty or a lack of order in the information 


available to the program manager. The measure of entropy is 


Do 


based on subjective, personal probability statements from 
experts in the field. 
6. Naval Underwater Systems Center, Cost/Schedule Growth 
Prediction Methodology (NUSC-TR-8435), Newport, RI, 
October 1989. 
The theme of this report was the occurrence of risk in 
research and development and initial production phases of the 
weapon system acquisition process. The probabilities of 


various risks are combined to quantify cost and schedule 


growth. 


B. SPECIFIC APPLICATIONS 


1. Mullineaux, R. W., and Yanke, M. A., Proposed 


Methodology for the Estimation of Jet Engine Costs in 
the Early Phases of the Weapons System Acquisition 


Process, Graduate Paper, Air Force Institute of 
Technology, Wright-Patterson AFB, OH, June 1976. 


This study found that in addition to performance 
variables, raw material related variables were highly 
correlated with system cost. It also investigated and 
recommended the use of confidence intervals when estimating 


system costs. 


2. Shishko, Robert, Technological Change Through Product 
Improvement in Aircraft Turbine Engines (R-1061-PR), 


The Rand Corporation, Santa Monica, Ca., May 1973. 

This work is a continuation and extension of Alexander 
and Nelson’s 1972 report which is described in Chapter III. 
The concept of technological advance is investigated for a set 
of engines which went through product improvement. The study 


found that these growth engines displayed less technological 
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advance and generally lower development costs than did newly 
designed engines. 
3. Hess, R. W., and Romanoff, H. P., Aircraft Airframe 


Cost Estimating Relationships: All Mission Types (N- 
2283/1-AF), The Rand Corporation, Santa Monica, Ca., 


December 1987. 

This work is a companion note to the 1987 work by the 
Same authors which is cited in Chapter IV. It provides the 
detailed results of the cost estimating relationships for 


aircraft airframes of all mission types. 


4. Blalock, C. D., Analysis of Schedule Determination in 


Software Program Development and Software Development 
Estimation Models, Graduate Paper, Air Force Institute 


of Technology, Wright-Patterson AFB, OH, September 

1988. 

This work analyzed five software development models 
and compared estimates to actual results. Cited were twelve 
factors which most commonly impact on software delivery 


schedules, the most important delay being in the requirements 


definition phase. 
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